Trajectory Methods for Neural Network Training

A new class of methods for training multilayer feedforward neural networks is proposed. The proposed class of methods draws from methods for solving initial value problems of ordinary differential equations, and belong to the subclass of trajectory methods. The training of a multilayer feedforward neural network is equivalent to the minimization of the network’s error function with respect to the weights of the network. To address this problem we solve the differential equation , where is the vector of network weights and is the gradient of the error function of the network. The solution of the above system of ordinary differential equations corresponds to the solution of the aforementioned minimization problem.